System for implementing sub-database replication

ABSTRACT

A method of implementing sub-table replication starts with the processor detecting an update to an entitlements table. The processor performs filtering of a data table based on the update to the entitlements table. The data table including an entitlements column. The processor detects an update to the entitlements column and performs incremental replication of the data table by causing a version-based replication to be executed. Other embodiments are also described herein.

CROSS REFERENCED TO RELATED APPLICATIONS

This claims priority to U.S. Provisional Patent Application Ser. No.63/032,163, filed May 29, 2020, the contents of which are incorporatedherein by reference in their entirety.

TECHNICAL FIELD

The present disclosure relates systems, methods, and devices fordatabases and more particularly relates to sub-database replication.

BACKGROUND

Databases are widely used for data storage and access in computingapplications. Databases may include tables having rows and columns thatinclude or reference data that can be read, modified, or deleted usingqueries.

In some instances, it may be beneficial to replicate database data inmultiple locations or on multiple storage devices. Replicating data cansafeguard against system failures that may render data inaccessible overa cloud network or may cause data to be lost or permanently unreadable.However, data replication across a network comprising various regionscan induce latency as well as cost.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

In the drawings, which are not necessarily drawn to scale, like numeralsmay describe similar components in different views. To easily identifythe discussion of any particular element or act, the most significantdigit or digits in a reference number refer to the figure number inwhich that element is first introduced. Some embodiments are illustratedby way of example, and not limitation, in the figures of theaccompanying drawings in which:

FIG. 1 is a diagrammatic representation of a networked environment inwhich the present disclosure may be deployed, in accordance with someexamples.

FIG. 2 illustrates a diagrammatic representation of the replicationcontroller implementing sub-database replication using schema filteringand table filtering, in accordance with some examples.

FIG. 3 is a diagrammatic representation of the replication controllerimplementing data sharing from the database to clients within the sameregion, in accordance with some examples.

FIG. 4 illustrates a diagrammatic representation of the replicationcontroller implementing table replication, in accordance with someexamples.

FIG. 5 illustrates a diagrammatic representation of the replicationcontroller implementing sub-table replication, in accordance with someexamples.

FIG. 6 illustrates a diagrammatic representation of the replicationcontroller implementing sub-table replication using entitlements appliedat runtime, in accordance with some examples.

FIG. 7 illustrates a diagrammatic representation of the replicationcontroller implementing sub-table replication using entitlements appliedat runtime, in accordance with some examples.

FIG. 8 illustrates a process 800 of implementing sub-databasereplication in accordance with one embodiment.

FIG. 9 illustrates the details of the operation 804 from FIG. 8 inaccordance with one embodiment.

FIG. 10 illustrates a diagrammatic representation of the replicationcontroller implementing sub-table replication using an entitlementscolumn within the table, in accordance with some examples.

FIG. 11 illustrates a diagrammatic representation of the replicationcontroller implementing sub-table replication using an entitlementscolumn within the table, in accordance with some examples.

FIG. 12 illustrates a diagrammatic representation of the replicationcontroller implementing sub-table replication using an entitlementscolumn within the table, in accordance with some examples.

FIG. 13 illustrates a process 1300 of implementing sub-databasereplication in accordance with one embodiment.

FIG. 14 illustrates the details of the operation 1308 from FIG. 13 inaccordance with one embodiment.

FIG. 15 is a diagrammatic representation of a machine in the form of acomputer system within which a set of instructions may be executed forcausing the machine to perform any one or more of the methodologiesdiscussed herein, in accordance with some example embodiments.

FIG. 16 is a block diagram showing a software architecture within whichthe present disclosure may be implemented, according to an exampleembodiment.

DETAILED DESCRIPTION

Databases can store enormous sums of data in an organized manner forproviders and clients across a networked environment. For example, aprovider can store data for a number of clients in a same database. Whenthe clients are in different geographical regions, the database may needto be replicated to provide access to the data to the clients.Replicating entire databases across the networked environment can becostly and cause much latency due to the size of the databases.Accordingly, there is a need to replicate the relevant portions of thedatabase to different regions.

Among other things, embodiments described in the present disclosureimprove the functionality of the database service system 100 byimplementing sub-database replication to different target regions in thenetwork. Using the sub-database replication, the database service system100 reduces the replication costs and replication latency by filteringnon-critical objects.

FIG. 1 is a diagrammatic representation of a networked environment inwhich the present disclosure may be deployed, in accordance with someexamples. The database service system 100 in FIG. 1 is a processingplatform that provides for database services. In one embodiment, thedatabase service system 100 can implement database replication orsub-database replication. Sub-database replication includes, forexample, schema replication, table replication, sub-table replication,etc.

The database service system 100 includes a database service manager 108that is accessible by multiple users via a network 110 (e.g., theInternet). The users can access the database service manager 108 usingclient device 102, client device 106, and client device 104,respectively. Database service manager 108 can support any number ofusers desiring access to data or services of the database service system100. The users of client devices 102, 104, 106 may include, for example,end users providing data storage and retrieval queries and requests,system administrators managing the systems and methods described herein,software applications that interact with a database, and othercomponents/devices that interact with database service manager 108.

The database service manager 108 may provide various services andfunctions that support the operation of the systems and componentswithin the database service system 100. The database service manager 108has access to stored metadata associated with the data stored throughoutdatabase service system 100. In some embodiments, metadata includes asummary of data stored in remote data storage systems (e.g., database112, database 116, database 114, etc.) as well as data available from alocal cache. Additionally, metadata may include information regardinghow data is organized in the remote data storage systems and the localcaches.

Database service manager 108 is further in communication with aplurality of data storage devices including database 112, database 116,and database 114 to perform various data storage and data retrievaloperations. Although three databases 112, 114, and 116 are shown in FIG.1, the database service system 100 is capable of including any number ofdata storage devices. In some embodiments, databases 112, 114, and 116are cloud-based storage devices located in one or more geographiclocations. For example, databases 112, 114, and 116 may be part of apublic cloud infrastructure or a private cloud infrastructure, or anyother manner of distributed storage system. Databases 112, 114, and 116may include hard disk drives (HDDs), solid state drives (SSDs), storageclusters, or any other data storage technology. Additionally, while notshown, the databases 112, 114, and 116 can be comprised in a storageplatform that may further include a distributed file system (such asHadoop Distributed File Systems (HDFS)), object storage systems, and thelike.

While the database service manager 108 and the databases 112, 114, 116are shown in FIG. 1 as individual components, each of the databaseservice manager 108 and the databases 112, 114, 116 may be implementedas a distributed system (e.g., distributed across multiplesystems/platforms at multiple geographic locations) or may be combinedinto one or more systems.

As shown in FIG. 1, the database service manager 108 includes areplication controller 118 that implements database replication orsub-database replication in the database service system 100, accordingto some embodiments.

Database replication involves replication of the entire primary database(e.g., database 112) to a secondary database (e.g., database 116). Indatabase replication, the database is the atomic unit of replicationsuch that it can be replicated in its entirety or not at all. The maindisadvantages of database replication include high cost and latencyassociated with replicating a large database in its entirety. Further,the users are not provided the opportunity to minimize their costs andlatency by selecting or excluding objects in the database forreplication. For example, some users want to exclude objects from theirdatabase from replication. In one implementation, the replicationcontroller 118 can restructure the database to be replicated so that thedesired units of replication match users' individual databaseboundaries.

Schema Filtering and Table Filtering

To provide further flexibility to the users, the replication controller118 can implement sub-database replication. FIG. 2 illustrates adiagrammatic representation 200 of the replication controller 118implementing sub-database replication using schema filtering and tablefiltering, in accordance with some examples.

Within each database, there are a number of layers including schemas andtables. A schema is a logical container in the database, and a table isanother container within the schema that has rows and columns. Schemasand tables can be represented as objects to allow for schema-level andtable-level filtering.

In this embodiment, the replication controller 118 receives from eachuser an inclusion list or an exclusion list or any combination thereof.The inclusion list can comprise a list of objects from the database tobe replicated and the target database associated with each object in thelist. The objects can be on a schema, a table, or any combinationthereof. The exclusion list can comprise a list of objects from thedatabase to be excluded from replication to a target database.

As shown in FIG. 2, the user that is a provider has accounts P1, P2, P3in regions A, B, and C respectively. Replication controller 118 receivesan inclusion list from the user's client device 102 that identifiesschema S2 and table T1 for replication to target database in account P2in region B, and identifies schema S4, and tables T3 and T4 forreplication to target database in account P4 in region C. In thisexample, replication controller 118 can also receive an exclusion listfrom the user's client device 102 that identifies table T2 that isincluded in schema S2 for exclusion from replication to target databasein account P2 in region B.

Alternatively, the replication controller 118 can receive an exclusionlist from the user's client device 102 that identifies schemas S1, S3,S4 and tables T2, T3, and T4 to be excluded from target database inaccount P2 in region B, and identifies schemas S, S2, S3, and tables T1,T2 to be excluded from target database in account P3 in region C.

The replication controller 118 then causes the replication of theobjects from the primary database in account P1 in region A based on theinclusion or the exclusion list to secondary databases in account P2 andP3 accordingly.

In one embodiment, the replication controller 118 generates areplication policy based on the inclusion or exclusion lists. Forexample, the replication controller 118 can apply a database replicationpolicy to restrict replication of schemas and tables by name using theexclusion lists. Alternatively, the replication controller 118 can applya database replication policy to allow replication of schemas and tablesby name using the inclusion lists. In one implementation, thereplication policy can be a new first-class database object that thecustomers can use to define the schemas and tables to replicate totarget accounts. The primary database can also have multiple policiesand multiple secondary databases.

Data Sharing

FIG. 3 is a diagrammatic representation 300 of the replicationcontroller 118 implementing data sharing from the database to clientswithin the same region, in accordance with some examples.

Some users in the database service system 100 are providers (e.g.,provider user) that maintain and share data with their clients (that arealso users of database service system 100). As shown in FIG. 3, the datacan be stored in a single table (e.g., data table) that includes acustomer identification (ID) (e.g., client_id) column. The table canalso include columns for name, date, time. The data for all the clients(e.g., client_id 1, 2, 3) of the provider user can be stored in thetable.

Referring to FIG. 3, the provider user is also provided with anentitlements table to manage the sharing of the data to their clients.The entitlements table includes columns for customer ID (e.g.,client_id) and consumer account name (e.g., consumer account) associatedwith the database service system 100. From the entitlements table inFIG. 3, the customer ID 1 and 3 (e.g., client_id 1 and 3) also haveaccounts with the database service system 100 as consumer accounts C1,C3, respectively.

Since consumer accounts C1 and C3 are in the same region as the provideraccount, the replication controller 118 does not need to replicate data.The replication controller 118 joins the data table with theentitlements table on the customer ID, using a secure view to cause therelevant data from the data table to be shared with each of the consumeraccounts. The replication controller 118 can receive queries from theconsumer accounts C1 and C3 for their data from the data table. Inresponse, as shown in FIG. 3, the replication controller 118 can selectand share the rows that are associated with customer ID 1 (e.g.,client_id 1) with the consumer account C1. Similarly, the replicationcontroller 118 can select and share the rows that are associated withcustomer ID 3 (e.g., client_id 3) with the consumer account C3.Accordingly, the replication controller 118 is able to generatepersonalized shares where a subset of the table rows can be shared witheach consumer account. In one example, the replication controller 118identifies the current consumer account that is querying and selects andshares the rows associated with the identified consumer account.

Table-Level Replication

When the consumers accounts are located in different regions from thedata table (e.g., primary table), the replication controller 118 mayreplicate the entire table to provider accounts in the different regionsand implement the personalized shares in FIG. 3 in each of the differentregions. FIG. 4 illustrates a diagrammatic representation 400 of thereplication controller 118 implementing table replication, in accordancewith some examples.

In FIG. 4, the provider user has a provider account P1 in the CentralRegion, where the data table (e.g., primary data table) is stored. Theprovider user is storing data for clients with customer IDs (e.g.,client_id) 1, 2, 3, 4 in the data table. As shown in the entitlementstable, client_id 1 has a consumer account C1, client_id 2 has twoconsumer accounts C2-W, C2-E, client_id 3 has a consumer account C3, andclient_id 4 has a consumer account C4. The entitlements table in FIG. 4also includes a column that associates the customer IDs (client_id) withthe provider accounts across different regions (e.g., provider accountP1, P2, P3).

Since at least one consumer account (e.g., C1, C2-E, C2-W, C3, C4) is inthe West region and in the East region, the replication controller 118replicates the entire data table to the provider account P2 in the Westregion and the provider account P3 in the East region. Once the datatable is in provider account P2, the replication controller 118 canshare the subset of the table rows in the secondary table in provideraccount P2 with each consumer account C2-W and C3 in the West region(e.g., personalized shares). Similarly, once the data table is inprovider account P3, the replication controller 118 can share the subsetof the table rows in the secondary table in provider account P3 witheach consumer account C2-E and C1 in the East region.

Since the consumer account C4 is in the same Central region as theprovider user and the (primary) data table, the replication controller118 can respond to query requests from consumer account C4 by sharingthe data rows associated with client_id 4 to consumer account C4.

The disadvantage of the table replication in FIG. 4 is that rows in theprimary data table that may not be needed in a region are beingreplicated in that region since replication controller 118 isreplicating entire tables between regions. This can entail higher costand higher latency.

FIG. 5 illustrates a diagrammatic representation 500 of the replicationcontroller 118 implementing sub-table replication, in accordance withsome examples. In this embodiment, the replication controller 118replicates the subset of table rows that are needed to be shared in agiven region to the provider account in that region. For example, thereplication controller 118 selects and replicates the subset of tablerows (client_id 2, 3) that are needed to be shared to consumers (e.g.,C3 and C2-W) to the provider account P2 in the West region. Similarly,the replication controller 118 selects and replicates the subset oftable rows (client_id 1, 2) that are needed to be shared to consumers(e.g., C1 and C2-E) to the provider account P3 in the East region.Accordingly, compared to the table replication in FIG. 4, the sub-tablereplication in FIG. 5 improves the cost and the latency.

Selecting the rows from the data table to be replicated is challengingbecause data in the primary data table can be materialized as contiguousunits of storage called micro-partitions. The table can be a collectionof micro-partitions. Each micro-partition is a file that containsbetween 50 MB and 500 MB of uncompressed data. The size of themicro-partition can be equal or less than 16 megabytes (MB). Groups ofrows in the table can be mapped into individual micro-partitionsorganized in columns.

Further, using a data manipulation language (DML), changes can be madeto the data in the data table. In some implementations, changes may bemade by way of any DML statement. Examples of manipulating data mayinclude, but are not limited to, selecting, updating, changing, merging,and inserting data into tables. When new data is inserted, databaseservice manager 108 creates a new micro-partition. When data is updated,database service manager 108 marks the micro-partition storing the datafor deletion and creates a new micro-partition for the updated data.

Given that many rows in a table are packed into these micro-partitions,in one embodiment, to perform row-level filtering, the replicationcontroller 118 opens each micro-partition to review whether the rowstherein are to be replicated for the different regions. This row-levelfiltering allows for precision but causes a slowdown in the replication.

Provider User Materialize Rows into Separate Objects

In another embodiment, the rows can be materialized into separate object(e.g., separate tables) such that new micro-partitions are created forthe rows that need to be replicated. In one implementation, the provideruser materializes the rows from the data table to be replicated into aseparate table. Specifically, the provider user can create a new tableand specify therein the rows from the (primary) data table to bereplicated. The rows in the new table are materialized as a newmicro-partition and the provider user requests that the replicationcontroller 118 replicate the new table. Since only the relevant rows arebeing replicated for a region, the replication costs and replicationlatency are lower. However, there is a higher storage cost becauseduplication of the data to be replicated is needed, there is higherExtract, Transform, Load (ETL) cost, and higher ETL latency becausetransformations are needed for each row of data to be stored in a newtable and then replicated, and the effort required from the providercustomer is high.

Replication Controller Materialize Rows into Materialized Views

In another implementation, to alleviate the effort required of theprovider user, the replication controller 118 can materialize the rowsfrom the data table to be replicated into materialized views.

A materialized view is a database object that includes final orintermediate results of a database query. The materialized view mayinclude a local cached copy of database data, a subset of rows orcolumns of a table, the result of a join, the result of an aggregatefunction, and so forth. Materialized views may be defined by a client orsystem administrator and may include any suitable information.Materialized views are commonly generated to aid in the execution ofspecific common queries.

A materialized view as disclosed in the present application is adeclarative specification of a persistent query result that isautomatically maintained and transparently utilized. In one example, amaterialized view includes a local copy of data located remotely or mayinclude a subset of rows and/or columns (may be referred to as a“partition” or “micro-partition”) of a source data table or join resultor may alternatively include a summary using an aggregate function.Materialized views are generated by way of materialization, where theresults of a query are cached similar to memorization of the value of afunction in functional languages. Materialized views improve performanceof expensive queries by materializing and reusing common intermediatequery results in a workload. Materialized views are utilized to improveperformance of queries at the expense of maintenance cost and increasedstorage requirements.

In one example, a materialized view can include a summary of the rowsfrom the data table that are relevant to the provider user or that areassociated with a query from the provider user. When the replicationcontroller 118 detects an update to the data table, the replicationcontroller 118 can update the materialized view accordingly. Forexample, when the update to the data table includes a newmicro-partition being inserted into the source table, the replicationcontroller 118 refreshes the materialized view by inserting the newmicro-partition into the materialized view. Further, when the update tothe data table includes deleting a micro-partition, the replicationcontroller 118 can compact the materialized view by removing the deletedmicro-partition from the materialized view. The data table can thus bereplicated as is. For example, the replication controller 118 canestablish and implement policies based on the replication targetaccount. Since only the relevant rows are being replicated for a region,the replication costs and replication latency are low, but storage costremains high and there is an added cost related to the materializedview.

Replication Controller Replicates Subset of Rows (Sub-Table Replication)

Rather than materializing the rows into separate new micro-partitions,in another embodiment, the replication controller 118 can replicate thesubset of rows based on the replication target account withoutduplicating the subset of rows as illustrated in FIG. 5.

Row-Level Filtering Based on Provider User Specified Column

In one embodiment, to replicate the subset of rows, the replicationcontroller 118 receives from the provider customer an identification ofthe column in a data table that is to be filtered and the filteringbehavior to be applied. For example, in FIG. 5, the provider customer P1can indicate that the customer ID column is to be filtered for client_id1 rows and to replicate the client_id 1 rows in provider account P3 inthe East region for consumer C1.

As discussed above, the replication controller 118 can perform row-levelfiltering by opening each micro-partition to review whether the rowstherein are to be replicated for the different regions. This row-levelfiltering allows for precision but causes the replication performance tobe slow.

File-Level Filtering Based on Provider User Specified Column

The metadata for each file (micro-partition) can indicate the minimumand the maximum values in each column of the file. Thus, for eachmicro-partition, the metadata indicates the minimum and maximum customerID (client_id). For example, a micro-partition having a minimum andmaximum client_id of 1 and 3 indicates a possibility of rows beingassociated with client_id 2. Without opening this micro-partition, thereplication controller 118 can replicate this micro-partition forclient_id 1, 2, and 3. By avoiding opening the micro-partitions, thereplication performance is faster but also less precise. Specifically,the replication controller 118 that implements file-level filtering maybe replicating unnecessary rows to some regions.

To improve on the precision, the replication controller 118 can clusterthe tables based on a column that is defining the distribution policy togenerate micro-partitions that are better sorted in that column.However, there is an added cost associated with the clustering.

Storing Provider User Specified Routing Policy

As discussed above, in addition to indicating the rows in the data tableto be replicated, the provider user can also indicate the routingpolicy. For example, in FIG. 5, the provider customer P1 can indicatethat the customer ID column is to be filtered for client_id 1 rows andto replicate the client_id 1 rows in provider account P3 in the Eastregion for consumer C1. As shown in FIG. 5, the entitlements table canbe a separate table where the provider user can specify the routingpolicy. In another embodiment, the entitlements table can also be storedas metadata to provide a faster access path to the routing policy. Forexample, the entitlements table can be a new first-class object forpersonalized sharing for defining the rows to which the consumers C1,C2, C3 that are entitled.

To implement the sub-table replication, the replication controller 118can use entitlements applied at runtime or use an entitlements columnwithin the (primary) data table that is maintained by the provider useror automated by the replication controller 118, or a combinationthereof.

Sub-Table Replication: Entitlements Applied at Runtime

FIG. 6 illustrates a diagrammatic representation 600 of the replicationcontroller 118 implementing sub-table replication using entitlementsapplied at runtime, in accordance with some examples.

In FIG. 6, the provider user has a provider account P1 in the Centralregion with a data table 604 with rows of data for their clients 1, 2,3, 4 (e.g., client_id 1, 2, 3, 4). The provider user indicates in theentitlements table 602 that client_id 1 is associated with consumeraccount C1 and provider account P3 in the East region. Accordingly, thischange in the entitlements table 602 indicates that provider user wantsto replicate from the Central region to the East region for consumeraccount C1.

In one example, the provider user's update to the entitlements table 602to indicate that the client_id 1 is associated with consumer account C1in the provider account P3 is a change in the entitlements table 602that is detected by the replication controller 118 and causes thereplication controller 118 to initiate the replication associated withthe change.

In another example, the replication controller 118 can receive a requestfrom provider account P3 for customer account C1. In this example, theprovider user can specify the provider account P3 for a customer accountC1.

Using the request, the replication controller 118 can (file-level)filter the micro-partition in the data table using the minimum andmaximum client_id in the micro-partition metadata to locate themicro-partitions to be replicated to provider account P3. Thereplication controller 118 replicates the located micro-partitions tothe provider account P3.

When the provider user updates the entitlements table, the replicationcontroller 118 needs to perform further sub-table replication. FIG. 7illustrates a diagrammatic representation 700 of the replicationcontroller 118 implementing sub-table replication using entitlementsapplied at runtime, in accordance with some examples.

In FIG. 7, the provider user indicates in the entitlements table 602that client_id 2 is associated with consumer account C2-E and provideraccount P3 in the East region. This indicates that a new customer wantsto have data replicated to the East region.

The replication controller 118 can perform incremental replication whichuses two mechanisms: version-based and full inventory.

In version-based replication, when a change is made to the data table,the replication controller 118 can detect the change to the data tableand perform the replication based on this new version of the data table.For example, performing version-based replication can includeidentifying the rows in the data table affected by the change andreplicating these rows in the data table to the target accounts (e.g.,provider accounts).

In full-inventory replication, when a change is made to an entitlementstable, the replication controller 118 can select all the rows in thedata table associated with the provider account identifiers in theentitlements table that are affected by the change.

Version-based replication is faster than full inventory. In oneembodiment, when the entitlements table 602 changes, the replicationcontroller 118 causes a full inventory at the next refresh to beexecuted for every target account (e.g., provider account). While theeffort required from the provider user is low, the replication costs andreplication latency associated with sub-table replication usingentitlements applied at runtime is high.

Process of Implementing Sub-Table Replication: Entitlements Applied atRuntime

Although the described flowcharts can show operations as a sequentialprocess, many of the operations can be performed in parallel orconcurrently. In addition, the order of the operations may berearranged. A process is terminated when its operations are completed. Aprocess may correspond to a method, a procedure, an algorithm, etc. Theoperations of methods may be performed in whole or in part, may beperformed in conjunction with some or all of the operations in othermethods, and may be performed by any number of different systems, suchas the systems described herein, or any portion thereof, such as aprocessor included in any of the systems. For example, the processes canbe performed by the replication controller 118 or a processor includedin the replication controller 118, or a processor in database servicemanager 108, or a combination thereof.

FIG. 8 illustrates a process 800 of implementing sub-databasereplication in accordance with one embodiment. At operation 802, aprocessor in the replication controller 118 detects a first update to anentitlements table 602. As shown in FIG. 6 and FIG. 7, the entitlementstable 602 can include entitlements table rows that are associated withclient identifiers, consumer account identifiers, and provider accountidentifiers. The entitlements table 602 can include entitlements tablecolumns storing the consumer account identifiers and the provideraccount identifiers. As illustrated in FIG. 6 and FIG. 7, the provideraccount identifiers identify provider accounts in a plurality ofgeographic regions. The first update can be associated with a firstentitlements table row of the entitlements table rows. The first updatecan include, for instance, a first client identifier of the clientidentifiers, a first consumer account identifier of the consumer accountidentifiers, or a first provider account identifier of the provideraccount identifiers.

At operation 804, the processor performs filtering of a data table 604based on the first update. As shown in FIG. 6 and FIG. 7, the data table604 can comprise data table rows associated with the client identifiersand including data content (e.g., event, date, payload, etc.). In oneembodiment, the processor receives a replication request from the firstprovider account that includes a first consumer account identifier.

At operation 806, the processor detects a second update to theentitlements table 602. Examples of the first update and the secondupdate to the entitlements table 602 includes adding a new entitlementstable row to the entitlements table, deleting one of the entitlementstable rows, or altering information included in one of the entitlementstable rows. In one example, altering information included in one of theentitlements table rows includes altering the client identifiers,consumer account identifiers, or provider account identifiers.

At operation 808, the processor performs incremental replication of thedata table 604 by causing a full inventory replication at a next refreshto be executed for provider accounts associated with the provideraccount identifiers in the entitlements table 602.

In one example, the processor performs filtering of the data table 604in operation 804 by performing file-level filtering. FIG. 9 illustratesthe details of the operation 804 from FIG. 8 in accordance with oneembodiment.

At operation 902, the processor identifies micro-partitions in aplurality of micro-partitions in the data table 604 having metadataassociated with the first client identifier. Each of themicro-partitions in the data table 604 can comprise one or more of theplurality of data table rows.

In one example, micro-partitions in the data table 604 comprisesmetadata including a minimum client identifier and maximum clientidentifier. In this example, the metadata is associated with the firstclient identifier when the first client identifier is within a rangeestablished by the minimum client identifier and the maximum clientidentifier.

At operation 904, the processor replicates the identifiedmicro-partitions to a first provider account associated with the firstprovider account identifier.

Sub-Table Replication: Entitlements Column within Table

In order to cause a version-based replication to be performed, at leastone change to the primary data table 604 is needed. For example, a tableDML can be triggered to cause the replication controller 118 to performversion-based replication. FIG. 10 illustrates a diagrammaticrepresentation 1000 of the replication controller 118 implementingsub-table replication using an entitlements column 1002 within thetable, in accordance with some examples.

In FIG. 10, the (primary) data table comprises an entitlements column1002 (e.g., target account column) that indicates the provider accountassociated with the row in the table. In one embodiment, the provideruser updates the column in the data table 1004 whenever the entitlementstable 602 change. The update to the entitlements column 1002 in the datatable causes the replication controller 118 to perform version-basedreplication. For example, the replication controller 118 can update thedata table 1004 version and create new micro-partitions to reflect thechange in the entitlements table 602. In this embodiment, thereplication controller 118 performing version-based replication includesreplicating rows in the data table 1004 to the target accounts (e.g.,provider accounts) using the entitlements column 1002.

FIG. 11 illustrates a diagrammatic representation 1100 of thereplication controller 118 implementing sub-table replication using anentitlements column within the table, in accordance with some examples.

In FIG. 11, the provider user updates the entitlements table 602 toindicate that client_id 2 is associated with consumer account C2-E andprovider account P3 in the East region. In this embodiment, the provideruser also updates the entitlements column 1002 in the data table 1004 toreflect that the row with client_id 2 is also associated with provideraccount P3.

As in FIG. 10, the update to the entitlements column 1002 in the datatable 1004 causes the replication controller 118 to performversion-based replication. In this embodiment, the replicationcontroller 118 performing version-based replication includes replicatingrows in the data table 1004 to the target accounts (e.g., provideraccounts) using the entitlements column 1002. As shown in both FIG. 10and FIG. 11, sub-table replication using an entitlements column 1002within the table lowers the replication cost and the replicationlatency, but there is higher ETL cost and ETL latency and the effortrequired from the provider user is much higher.

Sub-Table Replication: Virtual Entitlements Column within Table

To alleviate the effort required by provider user, in one embodiment,the replication controller 118 can maintain a virtual entitlementscolumn within the data table. FIG. 12 illustrates a diagrammaticrepresentation 1200 of the replication controller 118 implementingsub-table replication using an entitlements column within the table, inaccordance with some examples.

In FIG. 12, the (primary) data table comprises a virtual entitlementscolumn 1202 (e.g., target account column) that indicates the provideraccount associated with the row in the data table 1004. The virtualentitlements column 1202 is maintained by the replication controller118.

As shown in FIG. 12, the provider user updates the entitlements table602 to indicate that client_id 1 is associated with consumer account C1and provider account P3 in the East region and client_id 2 is associatedwith consumer account C2-E and provider account P3 in the East region.When the provider user updates the entitlements table 602, thereplication controller 118 populates the virtual entitlements column1202 in the data table 1004. In this embodiment, the replicationcontroller 118 updates the virtual entitlements column 1202 for the rowsof client_id 1 and 2 to indicate provider account P3. In response tothis update to the virtual entitlements column 1202 in the data table1004, the replication controller 118 can perform version-basedreplication. In this embodiment, the replication controller 118performing version-based replication includes replicating rows in thedata table 1004 to the target accounts (e.g., provider accounts) usingthe virtual entitlements column 1202.

While the ETL cost and ETL latency remains high, using the virtualentitlements column 1202 within the data table 1004 as shown in FIG. 12lowers the replication cost and the replication latency. Further, sincethe replication controller 118 maintains the virtual entitlements column1202, the effort level required from the provider user is reduced.

Process of Implementing Sub-Table Replication: Entitlements Columnwithin Table or Virtual Entitlements Column within Table

FIG. 13 illustrates a process 1300 of implementing sub-databasereplication in accordance with one embodiment. At operation 1302, theprocessor of the replication controller 118 detects an update to anentitlements table 602. As shown in FIG. 10, FIG. 11, and FIG. 12, theentitlements table 602 can include entitlements table rows that areassociated with client identifiers, consumer account identifiers, andprovider account identifiers. The entitlements table 602 can includeentitlements table columns storing the consumer account identifiers andthe provider account identifiers. As illustrated in FIG. 10, FIG. 11,and FIG. 12, the provider account identifiers identify provider accountsin a plurality of geographic regions. The update to the entitlementstable 602 can be associated with a first entitlements table row of theentitlements table rows. The update to the entitlements table 602 caninclude, for instance, a first client identifier of the clientidentifiers, a first consumer account identifier of the consumer accountidentifiers, or a first provider account identifier of the provideraccount identifiers.

Examples of the update to the entitlements table 602 in operation 1302includes adding a new entitlements table row to the entitlements table,deleting one of the entitlements table rows, or altering informationincluded in one of the entitlements table rows. In one example, alteringinformation included in one of the entitlements table rows includesaltering the client identifiers, consumer account identifiers, orprovider account identifiers.

At operation 1304, the processor performs filtering of a data table 1004based on the update to the entitlements table 602. As illustrated inFIG. 10, FIG. 11, and FIG. 12, the data table 1004 can comprise datatable rows associated the client identifiers and includes anentitlements column 1002 (or a virtual entitlements column 1202) anddata content (e.g., event, date, payload, etc.). As shown in FIG. 10,FIG. 11, and FIG. 12, the entitlements column 1002 (or virtualentitlements column 1202) can include the provider account identifiers.In one embodiment, the processor receives a replication request from thefirst provider account that includes a first consumer accountidentifier. In one example, the processor can perform file-levelfiltering in operation 1304 as detailed in FIG. 9.

At operation 1306, the processor detects an update to an entitlementscolumn 1002 (or virtual entitlements column 1202). For example, theentitlements column 1002, in FIG. 10 and FIG. 11, is populated orupdated by a provider user via a client device. The processor detectingan update to virtual entitlements column 1202 can further comprisedetermining that the update to the entitlements table 602 includes achange in one of the provider accounts identifiers in the entitlementstable 602 and populating the virtual entitlements column 1202 based onthe change in the one of the provider accounts identifiers in theentitlements table 602.

At operation 1308, the processor performs incremental replication of thedata table 1004 by causing a version-based replication to be executed.FIG. 14 illustrates the details of the process of version-basedreplication in operation 1308 from FIG. 13 in accordance with oneembodiment. At operation 1402, the processor updates a versionidentifier associated with the data table 1004. At operation 1404, theprocessor adds a plurality of micro-partitions in the data table 1004reflecting the update to the entitlements column 1002 (or virtualentitlements column 1202). At operation 1406, the processor replicatesthe data table rows to a plurality of provider accounts associated withthe provider account identifiers in the entitlements table 602 using theentitlements column 1002 (or virtual entitlements column 1202).

Machine Architecture

FIG. 15 is a diagrammatic representation of the machine 1500 withinwhich instructions 1510 (e.g., software, a program, an application, anapplet, an app, or other executable code) for causing the machine 1500to perform any one or more of the methodologies discussed herein may beexecuted. For example, the instructions 1510 may cause the machine 1500to execute any one or more of the methods described herein. Theinstructions 1510 transform the general, non-programmed machine 1500into a particular machine 1500 programmed to carry out the described andillustrated functions in the manner described. The machine 1500 mayoperate as a standalone device or may be coupled (e.g., networked) toother machines. In a networked deployment, the machine 1500 may operatein the capacity of a server machine or a client machine in aserver-client network environment, or as a peer machine in apeer-to-peer (or distributed) network environment. The machine 1500 maycomprise, but not be limited to, a server computer, a client computer, apersonal computer (PC), a tablet computer, a laptop computer, a netbook,a set-top box (STB), a personal digital assistant (PDA), anentertainment media system, a cellular telephone, a smartphone, a mobiledevice, a wearable device (e.g., a smartwatch), a smart home device(e.g., a smart appliance), other smart devices, a web appliance, anetwork router, a network switch, a network bridge, or any machinecapable of executing the instructions 1510, sequentially or otherwise,that specify actions to be taken by the machine 1500. Further, whileonly a single machine 1500 is illustrated, the term “machine” shall alsobe taken to include a collection of machines that individually orjointly execute the instructions 1510 to perform any one or more of themethodologies discussed herein. The machine 1500, for example, maycomprise the client device 102 or any one of a number of server devicesforming part of the Database service manager 108. In some examples, themachine 1500 may also comprise both client and server systems, withcertain operations of a particular method or algorithm being performedon the server-side and with certain operations of the particular methodor algorithm being performed on the client-side.

The machine 1500 may include processors 1504, memory 1506, andinput/output I/O components 638, which may be configured to communicatewith each other via a bus 1538. In an example, the processors 1504(e.g., a Central Processing Unit (CPU), a Reduced Instruction SetComputing (RISC) Processor, a Complex Instruction Set Computing (CISC)Processor, a Graphics Processing Unit (GPU), a Digital Signal Processor(DSP), an Application Specific Integrated Circuit (ASIC), aRadio-Frequency Integrated Circuit (RFIC), another processor, or anysuitable combination thereof) may include, for example, at least oneprocessor 1508 that execute the instructions 1510. The term “processor”is intended to include multi-core processors that may comprise two ormore independent processors (sometimes referred to as “cores”) that mayexecute instructions contemporaneously. Although FIG. 15 shows multipleprocessors 1504, the machine 1500 may include a single processor with asingle-core, a single processor with multiple cores (e.g., a multi-coreprocessor), multiple processors with a single core, multiple processorswith multiples cores, or any combination thereof.

The memory 1506 includes a main memory 1512, a static memory 1514, and astorage unit 1516, both accessible to the processors 1504 via the bus1538. The main memory 1506, the static memory 1514, and storage unit1516 store the instructions 1510 embodying any one or more of themethodologies or functions described herein. The instructions 1510 mayalso reside, completely or partially, within the main memory 1512,within the static memory 1514, within machine-readable medium 1518within the storage unit 1516, within at least one of the processors 1504(e.g., within the processor's cache memory), or any suitable combinationthereof, during execution thereof by the machine 1500.

The I/O components 1502 may include a wide variety of components toreceive input, provide output, produce output, transmit information,exchange information, capture measurements, and so on. The specific I/Ocomponents 1502 that are included in a particular machine will depend onthe type of machine. For example, portable machines such as mobilephones may include a touch input device or other such input mechanisms,while a headless server machine will likely not include such a touchinput device. It will be appreciated that the I/O components 1502 mayinclude many other components that are not shown in FIG. 15. In variousexamples, the I/O components 1502 may include user output components1524 and user input components 1526. The user output components 1524 mayinclude visual components (e.g., a display such as a plasma displaypanel (PDP), a light-emitting diode (LED) display, a liquid crystaldisplay (LCD), a projector, or a cathode ray tube (CRT)), acousticcomponents (e.g., speakers), haptic components (e.g., a vibratory motor,resistance mechanisms), other signal generators, and so forth. The userinput components 1526 may include alphanumeric input components (e.g., akeyboard, a touch screen configured to receive alphanumeric input, aphoto-optical keyboard, or other alphanumeric input components),point-based input components (e.g., a mouse, a touchpad, a trackball, ajoystick, a motion sensor, or another pointing instrument), tactileinput components (e.g., a physical button, a touch screen that provideslocation and force of touches or touch gestures, or other tactile inputcomponents), audio input components (e.g., a microphone), and the like.

In further examples, the I/O components 1502 may include biometriccomponents 1528, motion components 1530, environmental components 1532,or position components 1534, among a wide array of other components. Forexample, the biometric components 1528 include components to detectexpressions (e.g., hand expressions, facial expressions, vocalexpressions, body gestures, or eye-tracking), measure biosignals (e.g.,blood pressure, heart rate, body temperature, perspiration, or brainwaves), identify a person (e.g., voice identification, retinalidentification, facial identification, fingerprint identification, orelectroencephalogram-based identification), and the like. The motioncomponents 1530 include acceleration sensor components (e.g.,accelerometer), gravitation sensor components, rotation sensorcomponents (e.g., gyroscope).

The environmental components 1532 include, for example, one or cameras(with still image/photograph and video capabilities), illuminationsensor components (e.g., photometer), temperature sensor components(e.g., one or more thermometers that detect ambient temperature),humidity sensor components, pressure sensor components (e.g.,barometer), acoustic sensor components (e.g., one or more microphonesthat detect background noise), proximity sensor components (e.g.,infrared sensors that detect nearby objects), gas sensors (e.g., gasdetection sensors to detection concentrations of hazardous gases forsafety or to measure pollutants in the atmosphere), or other componentsthat may provide indications, measurements, or signals corresponding toa surrounding physical environment.

The position components 1534 include location sensor components (e.g., aGPS receiver component), altitude sensor components (e.g., altimeters orbarometers that detect air pressure from which altitude may be derived),orientation sensor components (e.g., magnetometers), and the like.

Communication may be implemented using a wide variety of technologies.The I/O components 1502 further include communication components 1536operable to couple the machine 1500 to a network 1520 or devices 1522via respective coupling or connections. For example, the communicationcomponents 1536 may include a network interface component or anothersuitable device to interface with the network 1520. In further examples,the communication components 1536 may include wired communicationcomponents, wireless communication components, cellular communicationcomponents, Near Field Communication (NFC) components, Bluetooth®components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and othercommunication components to provide communication via other modalities.The devices 1522 may be another machine or any of a wide variety ofperipheral devices (e.g., a peripheral device coupled via a USB).

Moreover, the communication components 1536 may detect identifiers orinclude components operable to detect identifiers. For example, thecommunication components 1536 may include Radio Frequency Identification(RFID) tag reader components, NFC smart tag detection components,optical reader components (e.g., an optical sensor to detectone-dimensional bar codes such as Universal Product Code (UPC) bar code,multi-dimensional bar codes such as Quick Response (QR) code, Azteccode, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2Dbarcode, and other optical codes), or acoustic detection components(e.g., microphones to identify tagged audio signals). In addition, avariety of information may be derived via the communication components1536, such as location via Internet Protocol (IP) geolocation, locationvia Wi-Fi® signal triangulation, location via detecting an NFC beaconsignal that may indicate a particular location, and so forth.

The various memories (e.g., main memory 1512, static memory 1514, andmemory of the processors 1504) and storage unit 1516 may store one ormore sets of instructions and data structures (e.g., software) embodyingor used by any one or more of the methodologies or functions describedherein. These instructions (e.g., the instructions 1510), when executedby processors 1504, cause various operations to implement the disclosedexamples.

The instructions 1510 may be transmitted or received over the network1520, using a transmission medium, via a network interface device (e.g.,a network interface component included in the communication components1536) and using any one of several well-known transfer protocols (e.g.,hypertext transfer protocol (HTTP)). Similarly, the instructions 1510may be transmitted or received using a transmission medium via acoupling (e.g., a peer-to-peer coupling) to the devices 1522.

Software Architecture

FIG. 16 is a block diagram 1600 illustrating a software architecture1604, which can be installed on any one or more of the devices describedherein. The software architecture 1604 is supported by hardware such asa machine 1602 that includes processors 1620, memory 1626, and I/Ocomponents 1638. In this example, the software architecture 1604 can beconceptualized as a stack of layers, where each layer provides aparticular functionality. The software architecture 1604 includes layerssuch as an operating system 1612, libraries 1610, frameworks 1608, andapplications 1606. Operationally, the applications 1606 invoke API calls1650 through the software stack and receive messages 1652 in response tothe API calls 1650.

The operating system 1612 manages hardware resources and provides commonservices. The operating system 1612 includes, for example, a kernel1614, services 1616, and drivers 1622. The kernel 1614 acts as anabstraction layer between the hardware and the other software layers.For example, the kernel 1614 provides memory management, processormanagement (e.g., scheduling), component management, networking, andsecurity settings, among other functionality. The services 1616 canprovide other common services for the other software layers. The drivers1622 are responsible for controlling or interfacing with the underlyinghardware. For instance, the drivers 1622 can include display drivers,camera drivers, BLUETOOTH® or BLUETOOTH® Low Energy drivers, flashmemory drivers, serial communication drivers (e.g., USB drivers), WI-FI®drivers, audio drivers, power management drivers, and so forth.

The libraries 1610 provide a common low-level infrastructure used by theapplications 1606. The libraries 1610 can include system libraries 1618(e.g., C standard library) that provide functions such as memoryallocation functions, string manipulation functions, mathematicfunctions, and the like. In addition, the libraries 1610 can include APIlibraries 1624 such as media libraries (e.g., libraries to supportpresentation and manipulation of various media formats such as MovingPicture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC),Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC),Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group(JPEG or JPG), or Portable Network Graphics (PNG)), graphics libraries(e.g., an OpenGL framework used to render in two dimensions (2D) andthree dimensions (3D) in a graphic content on a display), databaselibraries (e.g., SQLite to provide various relational databasefunctions), web libraries (e.g., WebKit to provide web browsingfunctionality), and the like. The libraries 1610 can also include a widevariety of other libraries 1628 to provide many other APIs to theapplications 1606.

The frameworks 1608 provide a common high-level infrastructure that isused by the applications 1606. For example, the frameworks 1608 providevarious graphical user interface (GUI) functions, high-level resourcemanagement, and high-level location services. The frameworks 1608 canprovide a broad spectrum of other APIs that can be used by theapplications 1606, some of which may be specific to a particularoperating system or platform.

In an example, the applications 1606 may include a home application1636, a contacts application 1630, a browser application 1632, a bookreader application 1634, a location application 1642, a mediaapplication 1644, a messaging application 1646, a game application 1648,and a broad assortment of other applications such as a third-partyapplication 1640. The applications 1606 are programs that executefunctions defined in the programs. Various programming languages can beemployed to create one or more of the applications 1606, structured in avariety of manners, such as object-oriented programming languages (e.g.,Objective-C, Java, or C++) or procedural programming languages (e.g., Cor assembly language). In a specific example, the third-partyapplication 1640 (e.g., an application developed using the ANDROID™ orIOS™ software development kit (SDK) by an entity other than the vendorof the particular platform) may be mobile software running on a mobileoperating system such as IOS™, ANDROID™, WINDOWS® Phone, or anothermobile operating system. In this example, the third-party application1640 can invoke the API calls 1650 provided by the operating system 1612to facilitate functionality described herein.

Glossary

“Carrier signal” refers to any intangible medium that is capable ofstoring, encoding, or carrying instructions for execution by themachine, and includes digital or analog communications signals or otherintangible media to facilitate communication of such instructions.Instructions may be transmitted or received over a network using atransmission medium via a network interface device.

“Client device” refers to any machine that interfaces to acommunications network to obtain resources from one or more serversystems or other client devices. A client device may be, but is notlimited to, a mobile phone, desktop computer, laptop, portable digitalassistants (PDAs), smartphones, tablets, ultrabooks, netbooks, laptops,multi-processor systems, microprocessor-based or programmable consumerelectronics, game consoles, set-top boxes, or any other communicationdevice that a user may use to access a network.

“Communication network” refers to one or more portions of a network thatmay be an ad hoc network, an intranet, an extranet, a virtual privatenetwork (VPN), a local area network (LAN), a wireless LAN (WLAN), a widearea network (WAN), a wireless WAN (WWAN), a metropolitan area network(MAN), the Internet, a portion of the Internet, a portion of the PublicSwitched Telephone Network (PSTN), a plain old telephone service (POTS)network, a cellular telephone network, a wireless network, a Wi-Fi®network, another type of network, or a combination of two or more suchnetworks. For example, a network or a portion of a network may include awireless or cellular network and the coupling may be a Code DivisionMultiple Access (CDMA) connection, a Global System for Mobilecommunications (GSM) connection, or other types of cellular or wirelesscoupling. In this example, the coupling may implement any of a varietyof types of data transfer technology, such as Single Carrier RadioTransmission Technology (1×RTT), Evolution-Data Optimized (EVDO)technology, General Packet Radio Service (GPRS) technology, EnhancedData rates for GSM Evolution (EDGE) technology, third GenerationPartnership Project (3GPP) including 3G, fourth generation wireless (4G)networks, Universal Mobile Telecommunications System (UMTS), High SpeedPacket Access (HSPA), Worldwide Interoperability for Microwave Access(WiMAX), Long Term Evolution (LTE) standard, others defined by variousstandard-setting organizations, other long-range protocols, or otherdata transfer technology.

“Component” refers to a device, physical entity, or logic havingboundaries defined by function or subroutine calls, branch points, APIs,or other technologies that provide for the partitioning ormodularization of particular processing or control functions. Componentsmay be combined via their interfaces with other components to carry outa machine process. A component may be a packaged functional hardwareunit designed for use with other components and a part of a program thatusually performs a particular function of related functions. Componentsmay constitute either software components (e.g., code embodied on amachine-readable medium) or hardware components. A “hardware component”is a tangible unit capable of performing certain operations and may beconfigured or arranged in a certain physical manner. In various exampleembodiments, one or more computer systems (e.g., a standalone computersystem, a client computer system, or a server computer system) or one ormore hardware components of a computer system (e.g., a processor or agroup of processors) may be configured by software (e.g., an applicationor application portion) as a hardware component that operates to performcertain operations as described herein. A hardware component may also beimplemented mechanically, electronically, or any suitable combinationthereof. For example, a hardware component may include dedicatedcircuitry or logic that is permanently configured to perform certainoperations. A hardware component may be a special-purpose processor,such as a field-programmable gate array (FPGA) or an applicationspecific integrated circuit (ASIC). A hardware component may alsoinclude programmable logic or circuitry that is temporarily configuredby software to perform certain operations. For example, a hardwarecomponent may include software executed by a general-purpose processoror other programmable processor. Once configured by such software,hardware components become specific machines (or specific components ofa machine) uniquely tailored to perform the configured functions and areno longer general-purpose processors. It will be appreciated that thedecision to implement a hardware component mechanically, in dedicatedand permanently configured circuitry, or in temporarily configuredcircuitry (e.g., configured by software), may be driven by cost and timeconsiderations. Accordingly, the phrase “hardware component” (or“hardware-implemented component”) should be understood to encompass atangible entity, be that an entity that is physically constructed,permanently configured (e.g., hardwired), or temporarily configured(e.g., programmed) to operate in a certain manner or to perform certainoperations described herein. Considering embodiments in which hardwarecomponents are temporarily configured (e.g., programmed), each of thehardware components need not be configured or instantiated at any oneinstance in time. For example, where a hardware component comprises ageneral-purpose processor configured by software to become aspecial-purpose processor, the general-purpose processor may beconfigured as respectively different special-purpose processors (e.g.,comprising different hardware components) at different times. Softwareaccordingly configures a particular processor or processors, forexample, to constitute a particular hardware component at one instanceof time and to constitute a different hardware component at a differentinstance of time. Hardware components can provide information to, andreceive information from, other hardware components. Accordingly, thedescribed hardware components may be regarded as being communicativelycoupled. Where multiple hardware components exist contemporaneously,communications may be achieved through signal transmission (e.g., overappropriate circuits and buses) between or among two or more of thehardware components. In embodiments in which multiple hardwarecomponents are configured or instantiated at different times,communications between such hardware components may be achieved, forexample, through the storage and retrieval of information in memorystructures to which the multiple hardware components have access. Forexample, one hardware component may perform an operation and store theoutput of that operation in a memory device to which it iscommunicatively coupled. A further hardware component may then, at alater time, access the memory device to retrieve and process the storedoutput. Hardware components may also initiate communications with inputor output devices, and can operate on a resource (e.g., a collection ofinformation). The various operations of example methods described hereinmay be performed, at least partially, by one or more processors that aretemporarily configured (e.g., by software) or permanently configured toperform the relevant operations. Whether temporarily or permanentlyconfigured, such processors may constitute processor-implementedcomponents that operate to perform one or more operations or functionsdescribed herein. As used herein, “processor-implemented component”refers to a hardware component implemented using one or more processors.Similarly, the methods described herein may be at least partiallyprocessor-implemented, with a particular processor or processors beingan example of hardware. For example, at least some of the operations ofa method may be performed by one or more processors orprocessor-implemented components. Moreover, the one or more processorsmay also operate to support performance of the relevant operations in a“cloud computing” environment or as a “software as a service” (SaaS).For example, at least some of the operations may be performed by a groupof computers (as examples of machines including processors), with theseoperations being accessible via a network (e.g., the Internet) and viaone or more appropriate interfaces (e.g., an API). The performance ofcertain of the operations may be distributed among the processors, notonly residing within a single machine, but deployed across a number ofmachines. In some example embodiments, the processors orprocessor-implemented components may be located in a single geographiclocation (e.g., within a home environment, an office environment, or aserver farm). In other example embodiments, the processors orprocessor-implemented components may be distributed across a number ofgeographic locations.

“Computer-readable storage medium” refers to both machine-storage mediaand transmission media. Thus, the terms include both storagedevices/media and carrier waves/modulated data signals. The terms“machine-readable medium,” “computer-readable medium” and“device-readable medium” mean the same thing and may be usedinterchangeably in this disclosure.

“Machine storage medium” refers to a single or multiple storage devicesand media (e.g., a centralized or distributed database, and associatedcaches and servers) that store executable instructions, routines anddata. The term shall accordingly be taken to include, but not be limitedto, solid-state memories, and optical and magnetic media, includingmemory internal or external to processors. Specific examples ofmachine-storage media, computer-storage media and device-storage mediainclude non-volatile memory, including by way of example semiconductormemory devices, e.g., erasable programmable read-only memory (EPROM),electrically erasable programmable read-only memory (EEPROM), FPGA, andflash memory devices; magnetic disks such as internal hard disks andremovable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks Theterms “machine-storage medium,” “device-storage medium,”“computer-storage medium” mean the same thing and may be usedinterchangeably in this disclosure. The terms “machine-storage media,”“computer-storage media,” and “device-storage media” specificallyexclude carrier waves, modulated data signals, and other such media, atleast some of which are covered under the term “signal medium.”

“Non-transitory computer-readable storage medium” refers to a tangiblemedium that is capable of storing, encoding, or carrying theinstructions for execution by a machine.

“Signal medium” refers to any intangible medium that is capable ofstoring, encoding, or carrying the instructions for execution by amachine and includes digital or analog communications signals or otherintangible media to facilitate communication of software or data. Theterm “signal medium” shall be taken to include any form of a modulateddata signal, carrier wave, and so forth. The term “modulated datasignal” means a signal that has one or more of its characteristics setor changed in such a matter as to encode information in the signal. Theterms “transmission medium” and “signal medium” mean the same thing andmay be used interchangeably in this disclosure.

What is claimed is:
 1. A method of implementing sub-table replicationcomprising: detecting, by a processor, an update to an entitlementstable, the entitlements table comprising a plurality of entitlementstable rows, the entitlements table rows being associated with aplurality of client identifiers, a plurality of consumer accountidentifiers, and a plurality of provider account identifiers, the updateto the entitlements table being associated with a first entitlementstable row of the entitlement table rows, the update to the entitlementstable including a first client identifier of the client identifiers, afirst consumer account identifier of the consumer account identifiers,or a first provider account identifier of the provider accountidentifiers; performing filtering of a data table based on the update tothe entitlements table, the data table comprising a plurality of datatable rows associated the client identifiers and including anentitlements column and data content, the entitlements column includingthe provider account identifiers; detecting an update to theentitlements column; and performing incremental replication of the datatable by causing a version-based replication to be executed.
 2. Themethod of claim 1, wherein performing incremental replication furthercomprises: updating a version identifier associated with the data table,adding a plurality of micro-partitions in the data table reflecting theupdate to the entitlements column, and replicating the data table rowsto a plurality of provider accounts associated with the provider accountidentifiers in the entitlements table using the entitlements column. 3.The method of claim 1, wherein the entitlements column is populated orupdated by a provider user via a client device.
 4. The method of claim1, wherein the entitlements column comprises a virtual entitlementscolumn.
 5. The method of claim 4, wherein detecting the update to theentitlements column further comprises: determining that the update tothe entitlements table includes a change in one of the provider accountsidentifiers in the entitlement table; and populating the virtualentitlements column based on the change in the one of the provideraccounts identifiers in the entitlements table.
 6. The method of claim2, wherein performing filtering further comprises: performing file-levelfiltering by: identifying micro-partitions from the plurality ofmicro-partitions in the data table having metadata associated with thefirst client identifier, each of the plurality of micro-partitions inthe data table comprising one or more of the plurality of data tablerows, and replicating the identified micro-partitions to a firstprovider account associated with the first provider account identifier.7. The method of claim 6, wherein each of the plurality ofmicro-partitions in the data table comprises metadata including aminimum client identifier and maximum client identifier.
 8. The methodof claim 7, wherein the metadata is associated with the first clientidentifier when the first client identifier is within a rangeestablished by the minimum client identifier and the maximum clientidentifier.
 9. The method of claim 2, wherein the plurality of provideraccount identifiers identify provider accounts in a plurality ofgeographic regions.
 10. The method of claim 2, wherein the entitlementstable further comprises a plurality of entitlements table columnsstoring the consumer account identifiers and the provider accountidentifiers.
 11. The method of claim 1, further comprising: receiving areplication request from the first provider account, the replicationrequest including a first consumer account identifier.
 12. The method ofclaim 1, wherein the update to the entitlement table includes adding anew entitlements table row to the entitlements table, deleting one ofthe entitlements table rows, or altering information included in one ofthe entitlements table rows.
 13. The method of claim 12, whereinaltering information included in one of the entitlements table rowsincludes altering the client identifiers, consumer account identifiers,or provider account identifiers.
 14. A system implementing sub-tablereplication comprising: a processor; and a memory having instructionsstored therein, when executed by the processor, causes the system toperform operations comprising: detecting an update to an entitlementstable, the entitlements table comprising a plurality of entitlementstable rows, the entitlements table rows being associated with aplurality of client identifiers, a plurality of consumer accountidentifiers, and a plurality of provider account identifiers, the updateto the entitlements table being associated with a first entitlementstable row of the entitlement table rows, the update to the entitlementstable including a first client identifier of the client identifiers, afirst consumer account identifier of the consumer account identifiers,or a first provider account identifier of the provider accountidentifiers; performing filtering of a data table based on the update tothe entitlements table, the data table comprising a plurality of datatable rows associated the client identifiers and including anentitlements column and data content, the entitlements column includingthe provider account identifiers; detecting an update to theentitlements column; and performing incremental replication of the datatable by causing a version-based replication to be executed.
 15. Thesystem of claim 14, wherein performing incremental replication furthercomprises: updating a version identifier associated with the data table,adding a plurality of micro-partitions in the data table reflecting theupdate to the entitlements column, and replicating the data table rowsto a plurality of provider accounts associated with the provider accountidentifiers in the entitlements table using the entitlements column. 16.The system of claim 14, wherein the entitlements column is populated orupdated by a provider user via a client device.
 17. The system of claim14, wherein the entitlements column comprises a virtual entitlementscolumn, and wherein detecting the update to the entitlements columnfurther comprises: determining that the update to the entitlements tableincludes a change in one of the provider accounts identifiers in theentitlement table; and populating the virtual entitlements column basedon the change in the one of the provider accounts identifiers in theentitlements table.
 18. The system of claim 15, wherein performingfiltering further comprises: performing file-level filtering by:identifying micro-partitions from the plurality of micro-partitions inthe data table having metadata associated with the first clientidentifier, each of the plurality of micro-partitions in the data tablecomprising one or more of the plurality of data table rows, andreplicating the identified micro-partitions to a first provider accountassociated with the first provider account identifier.
 19. The system ofclaim 18, wherein each of the plurality of micro-partitions in the datatable comprises metadata including a minimum client identifier andmaximum client identifier.
 20. The system of claim 19, wherein themetadata is associated with the first client identifier when the firstclient identifier is within a range established by the minimum clientidentifier and the maximum client identifier.
 21. The system of claim14, wherein the update to the entitlement table includes adding a newentitlements table row to the entitlements table, deleting one of theentitlements table rows, or altering information included in one of theentitlements table rows.
 22. A non-transitory computer-readable storagemedium including instructions that, when processed by a computer,configure the computer to perform operations comprising: detecting anupdate to an entitlements table, the entitlements table comprising aplurality of entitlements table rows, the entitlements table rows beingassociated with a plurality of client identifiers, a plurality ofconsumer account identifiers, and a plurality of provider accountidentifiers, the update to the entitlements table being associated witha first entitlements table row of the entitlement table rows, the updateto the entitlements table including a first client identifier of theclient identifiers, a first consumer account identifier of the consumeraccount identifiers, or a first provider account identifier of theprovider account identifiers; performing filtering of a data table basedon the update to the entitlements table, the data table comprising aplurality of data table rows associated the client identifiers andincluding an entitlements column and data content, the entitlementscolumn including the provider account identifiers; detecting an updateto the entitlements column; and performing incremental replication ofthe data table by causing a version-based replication to be executed.23. The non-transitory computer-readable storage medium of claim 22,wherein performing incremental replication further comprises: updating aversion identifier associated with the data table, adding a plurality ofmicro-partitions in the data table reflecting the update to theentitlements column, and replicating the data table rows to a pluralityof provider accounts associated with the provider account identifiers inthe entitlements table using the entitlements column.
 24. Thenon-transitory computer-readable storage medium of claim 22, wherein theentitlements column is populated or updated by a provider user via aclient device.
 25. The non-transitory computer-readable storage mediumof claim 22, wherein the entitlements column comprises a virtualentitlements column.
 26. The non-transitory computer-readable storagemedium of claim 25, wherein detecting the update to the entitlementscolumn further comprises: determining that the update to theentitlements table includes a change in one of the provider accountsidentifiers in the entitlement table; and populating the virtualentitlements column based on the change in the one of the provideraccounts identifiers in the entitlements table.
 27. The non-transitorycomputer-readable storage medium of claim 23, wherein performingfiltering further comprises: performing file-level filtering by:identifying micro-partitions from the plurality of micro-partitions inthe data table having metadata associated with the first clientidentifier, each of the plurality of micro-partitions in the data tablecomprising one or more of the plurality of data table rows, andreplicating the identified micro-partitions to a first provider accountassociated with the first provider account identifier.
 28. Thenon-transitory computer-readable storage medium of claim 27, whereineach of the plurality of micro-partitions in the data table comprisesmetadata including a minimum client identifier and maximum clientidentifier.
 29. The non-transitory computer-readable storage medium ofclaim 28, wherein the metadata is associated with the first clientidentifier when the first client identifier is within a rangeestablished by the minimum client identifier and the maximum clientidentifier.
 30. The non-transitory computer-readable storage medium ofclaim 22, wherein the update to the entitlement table includes adding anew entitlements table row to the entitlements table, deleting one ofthe entitlements table rows, or altering information included in one ofthe entitlements table rows.